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Abstract 

A method for preparing 3 -dimensional volumetric images of a selected organ for virtual endoscopy is provided. The 
patient is administrated either with an amount of contrast agent, or specific diet, or combination of both which will 
tag the residue in the lumen of the organ and allow the residue to be distinguished from surrounding tissues in the 
medical images. The patient's organ is imaged by computed tomography (CT), or magnetic resonance imaging 
(MRI), or other available modalities after the management of diet and contrast agent The acquired 3-dimensional 
volumetric images are then processed with tissue and organ segmentation and feature analysis focusing on the 
region of selected organ. The extracted feature data of clinical significant is transformed and fused with the original 
volumetric images for either removing residue in the lumen or facilitating the contrast between normal and abnormal 



1 Of 27 



i 



tissues. The fused volumetric images are displayed and rendered for physician inspection of abnormalities. 
Applications to laxative free CT virtual colonoscopy are provided. 
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FOOD GROUPS 


FOOD ALLOWED 


FOODS TO AVOID 


Breads 


White refined breads, rolls biscuits, 
muffins, crackers, pancakes and 
waffles 


Whole grain flower products of any type and 
baked goods made with bran, nuts, seeds, 
coconut, fruits, bagels, cornbread or graham 
crackers 


Cereals 


Refined cooked cereals, including 
cream of wheat, and farina, puffed 
wheat, puffed rice and rice krispies 


Oatmeal, any whole grained cereal bran or 
granola and any containing nuts, seeds, 
coconut or dried fruit. 


Desserts 


Plain cakes and cookies, water ices 
(Marinos), plain low-fat yogurt, 
Jello, custard, grape or apple jelly, 
plain hard candy, marshmallows 
and lite ice cream without nuts or 
chocolate 


Any desserts made with whole grain flour, 
bran, seeds, coconut, dried fruit, yogurts with 
fruit skins or seeds or nuts, sherbets and 
popcorn. No chocolate. 


Fruits 


Most canned or cooked fruits, 
applesauce and ripe banana. 


Dried fruits, all berries, most raw fruit, 
except banana. 


Potato and Potato 
Substitute 


Cooked white potato without the 
skin, white rice, white pasta and egg 
noodles 


All others, including whole-wheat pasta, 
noodles, vegetable pastas, and sweet potato. 


Vegetables 


Cooked fresh, frozen, or canned 
carrots, beets, asparagus tips, 
French style string beans and pureed 
squash, spinach. 


All others including raw and deep-fried 
vegetables, broccoli, cauliflower cabbage, 
spinach, sauerkraut, winter peas, corn, and 
any other vegetables with seed. 


Fats 


Margarine, salad oil, lite salad 
dressings, lite mayonnaise, and 
plain gravy. 


Butter, any fat containing whole grain flour, 
bran, seeds, nuts, Coconut, or dried fruit. 


Meats and Meat 
Substitutes 


Ground and well-cooked white meat 
chicken and turkey, with skin 
removed, Fish, shellfish, eggs, and 
low-fat cheese. 


Red meat, BBQ or pickled meat, any made 
with whole grain ingredients, seeds or nuts, 
dried beans, peas, lentils, legumes, peanut 
butter and whole milk Cheese. 


Soups 


Bouillon, broth, low-fat cream made 
with allowed vegetables, noodles, 
rice, or refined white flour. 


All others. 


Beverages 


Decaffeinated liquids of all kinds, 
Caffeinated beverages limited to 2-3 
(10 oz.) cups per day, low- fiat milk, 
and strained fruit juices. 


Espresso, frappucino, cappuccino, whole 
milk, fruit/vegetable juices containing pulp, 
prune juice, and all alcoholic beverages. 



Figure 4. 
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Foods in the provided Meal Kit 

Vanilla Nutritional Shakes 

Lemon drinks 
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Figure 5. 



Food Groups 


Food Allowed 


Foods to avoid 


Beverages 


Apple, Welch's white grape juices, 

Gatorade (no red, orange or purple) Decaffeinated weak tea. 


All other. 


Soups 


Clear Broths or Bouillon or consomme. 


All others 


Desserts ! 


Lemon or lime flavored Jell-O, water ices and frozen ice pops 


All others 



Figure 6. 
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Figure 11. 
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Figure 16. 
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Figure 12. This picture demonstrates an example of polyp rendering in different color in the 

virtual colonoscopy endoluminal view. 
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METHOD AND SYSTEM FOR PREPARING 3-DIMENSIONAL VOLUMETRIC IMAGES OF A 
SELECTED ORGAN FOR VIRTUAL ENDOSCOPY 



FIELD OF THE INVENTION 

The presented invention relates generally to a method or system for preparing 3-dimensional (3-D) volumetric 
images of a selected organ for virtual endoscopy. The virtual endoscopy is generally for early cancer detection 
related to organ with hollow lumen. One such application is for generating CT virtual colonoscopy images that does 
not require applying neither oral laxative nor suppository to the patient bowel preparation. 

BACKGROUND OF THE INVENTION 

Virtual endoscopy has received increasing attention as a niinimum-invasive and patient comfortable method for 
evaluating hollow organs. The fast increasing of virtual colonoscopy practice is an evidence that the virtual 
endoscopy approach is getting better acceptance. The basic idea of this technology is to fill the selected organ with 
materials that can be distinguished from the wall of the organ in the medical images. The acquired medical images 
are further processed for constructing either 2-D or 3-D model of the organ. The model is displayed in different 
rendering modes for physician's inspection of abnormalities (US Patents 5,782,762 and 5,971,767). 

The hollow organ may not show any cavity in the medical images if it is empty. For example, the empty colon and 
bladder usually collapse and show no cavity in CT images. Even if the lumen of organ is filled with some material, 
the lumen may still not be able to be distinguished from surrounding tissue. For example, the bladder that is filled 
with urine may not be distinguished well from surrounding tissue in CT images. Furthermore, there may be more 
than one kind of materials inside the lumen of the organ. For example, there are usually air, stool, and fluid in the 
human colon. That results more difficulties to detect abnormality related to the organ. Hence, for virtual endoscopy 
applications, it is necessary to manage either a certain amount of contrast agent (oral intake, suppository, or 
intravenous injection), or diet, or both of them to the patient for achieving optimal imaging results. 

The procedure of the administration of both contrast agent and diet is called organ preparation. The organ 
preparation may take several days or several minutes. The main concern is to achieve optimal tissue contrast in the 
images. Another concern about the organ preparation is non-invasive and patient comfortable. For example, there 
have been evidences showed that the major complaint to the optical colonoscopy is the bowel cleansing rather than 
the catherter procedure itself. Although the virtual colonoscopy is free from perforation, the physical cleansing 
bowel preparation procedure is one of the major barrier to allow virtual colonoscopy to be a screening tool for 
colorectal cancer. It is highly desirable for an easy handle, non-invasive, and patient comfortable bowel preparation 
for virtual colonoscopy. 

There are usually several hundreds of axial images for a single virtual endoscopy scan. It is a time-consuming work 
to review all those axial images. The virtual endoscopy usually provides a 3-D view of the inner surface of the 
organ. It is called endolurninal view. The physician can either change viewing and lighting source and direction or 
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automatically navigate inside the organ. The 3-D display organizes and shows the anatomy information in the slice 
images in a more efficient way. There have already been a lot of approaches for displaying and rendering the virtual 
endoscopy volumetric images (eg. US Patent, 5,971,767, US Patent 6,331,1 16, US Patent 6,477,401, and US Patent 
6,514,082). Those approaches basically focus on the mode of navigation inside the lumen and the speed of the 
rendering. They are efficient for shape description. If there is an irregular bump shooting out from the inner surface 
of the organ (like the polyp in the air-distended colon), it is easily detected in the virtual endoscopy. However, there 
are abnormalities happening right in the wall of the organ and resulting only mild shape deformation on the inner 
surface, especially for lesion at the early stage (like the bladder lesion and cancer). As in the MRI virtual cystoscopy 
images, the lesion that invading the wall of the organ demonstrate different texture comparing to the normal wall. If 
the texture information may be overlaid on the endoluminal view, it will greatly facilitate the early detection of 
lesion and cancer. If the texture information may be fused in the original volumetric images in an appropriate way, it 
is possible that the lesion region will be enhanced in the endoluminal view. Hence, the method for fusion volumetric 
images and feature data is highly desirable. This fusion technology should also be very helpful for displaying 
computer assisted lesion detection result in its real anatomical context 



SUMMARY OF THE INVENTION 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a flowchart representing a method in accordance with the present invention of preparing volumetric 
images of a selected organ for virtual endoscopy. 

Figure 2 is a flowchart representing a method and system of laxative free virtual colonoscopy. 

Figure 3 is a chart illustrating in detail the administration schedule and doses of the contrast agent for laxative free 

virtual colonoscopy. 

Figure 4 is a chart describing the suggested lower residue diet It tells the patient what kind of food should be 

avoided and what kind of food can be taken during the bowel preparation period. 

Figure 5 is a chart describing the foods that the Meal Kit provides. 

Figure 6 is a chart describing the foods that can be taken for the fluid diet 

Figure 7 is a comparison chart of those three diets for laxative free virtual colonscopy. 

Figure 8 is a chart to list tagging quality of those three bowel preparations for laxative free virtual colonoscopy. 

Figure 9 demonstrates electronic cleansing results on virtual colonoscopy images. The tagged residue (bright 

region) that is shown in the left panel in the colon lumen is removed in the right panel (becoming dark as air region). 

The layer between tagged residue and surrounding tissue in the left image is replaced by the layer between air and 

tissue in the right image. The region in the circle demonstrates a tiny tagged stool attaching on the colon wall. The 

tagged residue puddles mat are arrowed show the non-uniform tagging. 

Figure 10 is a flowchart describing steps of the electronic cleansing method. 
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Figure 11 shows a CT image that the colon wall has the similar intensity value as that of tagged residue. The 
arrowed 719 HU shading region is a thin haustral fold The arrowed brighter region of 823 HU is a puddle of tagged 
fluid in the colon. 

Figure 12 a chart that describes the 5 directional gradients for extraction of tissue boundary in CT images. 

Figure 13 is a chart showing the neighbor voxels that are used in computation of selected directional gradients for 

CT images. 

Figure 14 demonstrates the maximum gradient image (right) of a CT axial image (left). 

Figure 15 demonstrate the result of electronic cleansing in both 2-D and 3-D display. The top left image is the 
original CT slice image. The thin layer of tagged stool attaching to the ceiling of the colon is encircled The top right 
image shows the result of electronic cleansing. The bottom left image is the 3-D view of the colon wall that is 
covered by the thin layer of tagged stool. The bottom right image is the 3-D view of the colon wall after removal of 
the thin layer of tagged stool. It is obvious that the small thin haustral folds are better demonstrated in the cleansed 
3-D display. 

Figure 16 lists the quality evaluation results of the invented electronic cleansing algorithm. The result is based on 
the radiologist's reviewing of 60 cases from 35 healthy volunteers. 

Figure 17 demonstrates result of fusion between the computer assisted poly detection and the original volumetric 
images. The detected polyp shows different color from that of normal colon wall in the endoluminal view. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention generally relates to a method and system, as schematically represented in Fig. 1 , for preparing 
3-D volumetric images of a selected organ for virtual endoscopy. While die methods and system described in this 
application can be applied to any object with hollow lumen to be examined, the preferred embodiment is an body 
organs with hollow lumen such as colons, tracheo-bronchial airways, bladder and the like. 

The following is the detail description of the invented technologies for laxative free virtual colonoscopy. 

The method and system for laxative free virtual colonoscopy is described schematically in Fig.2. It illustrates major 
steps to perform a laxative free virtual colonoscopy. The result of polyp detection represents all suspicious regions 
or areas in the images that are detected automatically by algorithms. Whether the result of polyp detection should be 
fused with the original volumetric images depends of the decision of physician who uses the virtual colonoscopy. If 
the result of polyp detection is not available, the physician can still use the virtual colonoscopy system for polyp 
detection. 

The preferred embodiments including design of low residue diets and administration of doses of contrast agents for a 
laxative free bowel preparation for virtual colonoscopy that provides good residual stool and fluid tagging quality. 
The preferred embodiment also including intelligent algorithms for electronic cleansing of tagged residual stool and 
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fluid in the virtual colonoscopy images. The electronic cleansed volumetric images are then displayed for physician 
inspection. 

We use barium sulfate (2.1%, 250 ml) and Gastroview (diatrizoate meglumine and diatrizoate sodium solution, 367 
mg/ml, 120 ml) as colon residue contrast agent. The barium sulfate is in banana flavor. The Gastroview can be 
mixed with soft drink, like soda, if the patient feels more easily to take. The optimal dosage to administrate the 
contrast agent is the 36 hours 6 doses as described in the Fig. 3. Neither oral laxative nor suppository is given to the 
patient There are several kinds of barium and sodium iodine available in the market They result similar tagging 
quality with the same dosage. Hence, the invention is not limited to specific brand named contrast agent 

Without applying laxative, the stool and fluid residue may remain in the colon anywhere. The patient is required to 
follow specific diets within 3 days in order to facilitate uniform residue tagging and to mitigate the stool sticking 
around the colon wall. The design of diets focuses on easy handling and less interfering of patient's normal daily 
life. There were three kinds of three-days-diet being designed. The first diet (denoted as Diet 3) was a general lower 
residue diet. It was very close to the normal diet. If patient follows Diet 3, he/she is only to avoid several kinds of 
food that possibly result more residue for 3 days prior to imaging. The detail food list for the Diet 3 is shown in the 
Fig. 4. The second diet (denoted as Diet 2) let the patient following the same food list in day 3 and 2 prior to the day 
of scan and required the patient eating food only from a provided Meal Kit in the day 1 prior to the day of scan. The 
foods that provided in the Meal Kit are shown in the Fig. 4. The third diet (denoted as Diet 1) required the patient 
following a liquid food diet in the day 1 prior to the scan. In the other 2 days, the requirement of the Diet I is the 
same as that of the Diet 3. The foods for the fluid diet are shown in the Fig. 6. The Fig. 7 is a comparison matrix for 
the selected three kinds of diets. During the 3-days diet for all those 3 kinds of diets, the patient was suggested to 
take small portion of meal and avoid the following foods: whole grain flour or cereals, dried or raw fruits, raw or 
deep fried vegetables, tough fibrous meats, caffeinated liquids, nuts and seeds, and yogurt. The bowel preparation 
procedure (BP) for virtual colonoscopy combines one of the 3-day diets and the administration of 6 doses contrast 
agent. The bowel preparation procedure associated Diet 1, 2, and 3 are denoted as BP 1, 2, and 3 respectively. 

Prior to the scan, approximate 2 to 3 liters room air or C02 was forced into patient's colon in order to expand it to 
make the colon easier to scan and exam. The routine CT virtual colonscopy scanning protocol was utilized. An 
example of a helical CT scanner (single slice) would use 3 to 5 mm collimation, 1:1 to 2:1 pitch, 120 kVp, and 100 
mA. The patient was required to hold a breath. The scan usually took 35 to 45 seconds on the helical CT scanner 
depending on the body size of patient It generated 300 to 550 images for each scan series. The images were packed 
to form a volumetric dataset for further processing or reviewing. Usually, the patient was scanned twice in different 
body positions at a single visit to scanning room. One time the patient laid down with face up (supine position) and 
another rime the patient laid down with face done (prone position). 

The effectiveness of the bowel preparations was evaluated on 45 volunteers (average age 36, standard deviation 15, 
minimum 18, maximum 69, 6 females and 39 males). Each volunteer took at least one of the bowel preparations and 
underwent CT in one day. Some of them took two or all 3 of those bowel preparations and underwent CT scans in 
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different days (at least one week later). The acquired images were reviewed slice by slice by an experienced 
radiologist to evaluate the tagging quality. If intensity of residue voxel that was less than 150 Hounsfield (HU) Units 
was found in the images, the residue was defined as poorly tagged. If one region of residue that was larger than 5 
mm in diameter was found poorly tagged in the volumetric images, the dataset or the case related to the volumetric 
images was defined as poor tagged. Fig. 8 showed the results of tagging quality on those volunteer datasets. There 
were 79 datasets in total. Among them, there were 30 from BP 1, 35 from BP 2, and 14 from BP 3. From the Fig. 8 
we can conclude that the BP 1 achieved the best tagging quality. 

The goal of electronic cleansing is to detect and remove all tagged residue inside colonic lumen and to rebuild the 
image of boundary between tagged residue and colon wall into the image of boundary between air and colon wall. 
The Fig. 9 demonstrates a result of electronic cleansing. There are a lot of factors that make the electronic cleansing 
a tough task for laxative free virtual colonscopy CT images. First, the intensity range of tagged residue is similar to 
that of bone in CT images. The tagged residue in the rectum and sigmoid may frequently touch the backbone in 
supine position. That makes the separation of tagged residue from backbone difficult. Second, the residue may also 
be tagged in a non-uniform way so that a parameter set for cleansing one puddle successful may fail for cleansing 
another. Third, some of thin haustral fold submerged in the tagged residue may have very similar intensities that 
other tagged residue has due to partial volume effect of CT imaging. The previous inventions in the US Patent 
6,331,116, US Patent 6,343,936, and US Patent 6,514,082 provided electronic cleansing algorithms for virtual 
colonoscopy CT images. However, it was assume that the residue was tagged uniformly and formed a puddle with 
flat level due to the gravitation. The technologies there could not cleanse tagged stool attaching around the colon 
wall that does not form a puddle. Unfortunately, the stool often attaches around the colon wall if no laxative or 
suppository is applied. In this invention, new methods were presented to achieve better quality of electronic 
cleansing for any kinds of residue tagging condition and any morphology of residue region. 

The presented electronic cleansing method was composite of several steps. Fig. 10 shows a flowchart that 
representing those steps. The first step of electronic cleansing was the extraction of the colon region. In the laxative 
free virtual colonoscopy images, the colon region was composite of air, residue, and wall. The thickness of colon 
wall was within a certain range, e.g. less than 5 mm. If the colon lumen, including both air and residue, was found, 
the colon wall region could be detennined by dilation the region of colon lumen. In the CT images, intensity of bone 
is similar to that of tagged residue and the lung is also filled of air. In US Patent 6,331,116, US Patent 6,343,936, 
and US Patent 6,514,082, methods had been provided for detecting and removing the regions of both lung and bone 
while keeping and labeling the region of colon lumen. In this invention, the detection, cleansing, and boundary 
reconstruction of the region of tagged residue are the focuses. 

When all tagged residue regions were found, the low-level voxel classification algorithm that invented in the US 
Patent 6,33 1,1 16 was applied to all regions of tagged residue. The voxels in those regions were grouped into several 
clusters. Usually, there were 10 clusters being generated. The voxels with similar intensity properties were in the 
same group. For example, voxels that located in the center of uniform tagged region were in the same cluster and 
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voxels that located in the edge of an uniform tagged region were in another cluster that was different from that of the 
center cluster. However, the classification results basically represented intensity properties of the entire volumetric 
images. The voxels in the same cluster might represent different clinical meaning due to be in different tagged 
regions with different tagged conditions. For example, the intensity of the fold that was covered by tagged fluid 
might have similar intensity of tagged fluid due to partial volume effect Hence, electronic cleansing algorithm must 
be able to self-adaptive to variation of tagging condition in different tagged regions. 

The basic idea of die new algorithm was based on the following fact. In virtual colonoscopy images, the main 
concern was to detect the boundary between different tissues or materials. In laxative free virtual colonoscopy 
images, the concern was to detect and extract boundaries between air, colon wall, and tagged residue. Although the 
intensities of air, colon wall, and tagged residue were in separate ranges that separate from each other, the 
boundaries of them in the CT images could not be extracted using the simple thresholding due to non-uniform 
tagging and partial volume effect in the CT images. For example, the colon wall tissue often has intensity around [- 
100, 50] HU and the well-tagged residue has intensity around [200, 800] HU. If a thin haustral fold submerges in 
well-tagged residue puddle, the intensity of the colon wall of the thin fold may greater than 600 HU as die example 
shown in the Fig. II. The immediate solution was the edge detector. Since most edge detection technologies were 
based on the difference of intensity rather than the intensity range itself, they were good candidate for determining 
the boundaries. However, the conventional edge detection technology could not be applied directly here. First, the 
volumetric CT images often have anisotropic voxel size. Of course, the images can be re-sampled or interpolated 
into isotropy voxel size. But that takes extra computation time. Second, the task for electronic cleansing is to 
extraction of the layer of the boundary rather than line of edge curve. The layer of the boundary represents die 
partial volume effect, which comes from the physical limitation of CT scanner. Third, it is well known that most 
edge detection method is sensitive to noise and the noise level of CT images usually is high. 

In the current invention, a boundary layer extraction method was provided which was based on the maximal gradient 
approach. The gradient is defined as the directional first-order derivative. Usually, the integer grid of medical 
images has different unit lengths along different coordinate directions due to the variation of scanning protocols. 
When computing the discrete first-order derivative in the digital images, the anisotropic voxel size must be 
considered. Another concern is the so-called partial volume effect. The tissue boundary usually forms a thick layer 
with certain range due to the partial volume effect rather than a sharp line of curve. The intensity of voxels in the 
boundary layer changes from one type to another. The boundary layer should include more information on tissue 
abnormality or tissue feature rather than that of a sharp boundary line does. The maximal gradient approach is 
design to meet these purposes. The basic idea of the approach is to selected several discrete directions in the 3-D 
image grid. These directions should be uniformly covered all directions. Then, compute first-order derivatives along 
all those directions for each voxels and determine the maximum of the absolute value of those first-order derivatives 
associated. The maximum value is denoted as the gradient feature value (GFV) of that voxel. By thresholding the 
GFV, the boundary layer can be determined. 
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For simplicity, we describe the algorithm with a CT image that has voxel size around X : Y : Z = 0.7 : 0.7 : t.O mm. 
For each voxel in the image, we compute five directional derivatives. Fig. 12 shows the five directions. The X, Y, 
and Z are the orthogonal axial directions of the image grid coordinate system The otheT two directions are the 
diagonal directions on the X-Y plane. In this invention, the directional derivative computation is designed for the 
spiral CT images, where the Z-direction often has longer step length than those of X and Y-directions. If CT images 
are acquired using multi-slice CT with isotropic voxel size, the derivatives along diagonal directions in die X-Z and 
Y-Z planes must be also considered For facilitating clarity, the illustrating example is provided in the Fig. 12. 

The calculation of first-order derivative is described in the following equations. Let X y be the intensity of i-th 
neighbor voxels and denote K= fa: i = 1, 2, ... , 5} the vector of directional derivatives. Expression 1 shows how to 
calculate the Y. The neighborhood is defined in Fig. 13. 

y x =| x I2 - 18x 4 + 18ar 6 - x 14 | /(12 • ^) , y 2 =\ - 18*, + 18x 3 - * n | /(12 • X y ) 
ys=l*i6-*i7|/(2*AJ 

where X* is the step length along direction "*." In the Equation 1, the central-5-point formula is employed to 
compute the first-order difference. However, the invention is not solely limited to it Any other formula can be 
utilized for calculation of the first-order derivative in accounting of voxel size for any other applications. 

The GFV is determined as min{y h i - 1,2X4,5}. The GFV is much less sensitive to the image noise than value of 
single directional derivative. Fig. 14 shows an example of GFV image for a virtual colonoscopy CT axial image. 

By thresholding the GFV, the layer of boundary can be separated from the tissue homogeneous regions since the 
tissue homogeneous region has much lower GFV comparing to the tissue boundary where the intensity dramatically 
changes. The threshold can be pre-set for a specific application. As in the present invention for virtual colonoscopy 
CT images, the threshold can be set as 56. In other words, if the GFV of voxel is larger than 56, it is boundary 
voxels; Otherwise, the voxel is in a homogeneous tissue region. Usually, the nonuniform tagging in a region of 
tagged residue is slowly varying in intensity. That kind of intensity change does not contribute much to the value of 
GFV. That's why the GFV threshold cannot be too small. However, the threshold cannot be too large either since 
the extracted layer of boundary may not enclose the entire colon lumen. This is the result of non-uniform range of 
partial volume effect along different directions. If we reduce the threshold for GFV thresholding, we get better 
chance to enclose the colon lumen. However, that takes risk of over estimating the layer of boundary. To solve this 
problem, we fused the thresholded GFV result with the low-level voxel classification results. In other words, the 
information from both voxel classification and GFV are combined to achieve optimal estimation of layer of 
boundary. If voxel's GFV is larger than the preset threshold or the voxel is at the edge of region where all voxels are 
in the same cluster, the current voxel is a real boundary voxel. 



18 of 27 



The real boundary voxels form a boundary layer between air, colon wall, and tagged residue. The next step is to 
remove the tagged residue and to transform the boundary layer between colon wall and tagged residue into the 
boundary layer between air and colon wall. We call this procedure Boundary Layer Reconstructioa The 
reconstruction has to be implemented on each spatial separate region of tagged residue independently. In other 
words, the reconstruction method is the same, but the parameters of the method have to be adaptively adjusted to 
each residue region since each region of residue is in different tagging conditions. 



For a selected region of tagged residue, the intensity of voxel that is not in the boundary layer will be set to the 
average air intensity of CT images, e.g. -850 HU. The average air intensity of CT image is denoted as T air . There are 
three kinds boundary voxels in the tagged residue region. The first is the air/residue boundary. The second is the 
wall/residue boundary. And the third is the boundary voxel that there are all three kinds of voxels around it The first 
kind of boundary point is easy to detect It is close to the air region and has greater GFV (e.g. GFV is larger than 
500) since the intensity change from air to tagged region is the most dramatically in the CT image. When the region 
of the first kind of layer is determined, a conditional region growing algorithm will applied to the region. The region 
growing will expand the region into both tagged residue and air region. An example of the 6-connected 3-D region 
growing algorithm is described in the following: 



Push the seed in the Queue 
While ( Queue is not empty ) 

( 

Get the front voxel in the queue and denote its intensity as V c . 

Label the current voxel as in the region. 

Check the 6 closest 3-D neighbors of the current voxel. 

{ 

If the neighbor is not labeled as in the region, denote the neighbor intensity as v„. 

{ 

If { (V c > 150 HU) && <V c +50 HU S V„) M ( V„ < 350 HU ) 
{ 

Add this neighbor into the Queue 

} 

Else if ( (V e < -100 HU) && (Ve-50 HU 2 V„) && (V„ > T» lr ) ) 
{ 

Add this neighbor into the region 

) 

} // End of checking this neighbor 
} // find of checking all neighbors 

pop the front voxel from the Queue. 

} // While () 



The expanded region of the first kind of layer will cover the area of partial volume layer right between the tagged 
residue and the air lumen. Intensities of voxels in the expanded region will be set to T^ also. The expended region 
may also cover part of the layer of the third kind. The last step is to transform the layer of the 2 nd and the 3 rd into 
layer of air/tissue. Denote T tvc the average intensity of all voxels in the layer of the 2 nd and the 3" 1 . Denote Td^ the 
average intensity of soft tissue around colon lumen (e.g. -50 HU). Denote V c the intensity of the current voxel in the 
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layer of 2 nd and 3 rd and V f the transformed intensity value of the current voxel. The intensity transformation is 
defined by the following equation. 

If( V c > T ave ) 
{ 

V f = T elr ; 

) 

Else 
{ 

y = T tizsve ~ T air , ,j _ y x j 
I ave 1 tissue 

) 

After applying the transformation, the electronic cleansing on this region of tagged residue has been done. The Fig. 
15 demonstrates a typical result of electronic cleansing. As shown in the picture, the thin layer of the attaching stool 
is also cleansed well. 

We recruited 35 healthy male volunteers for evaluation of electronic cleansing method with laxative free bowel 
preparation. Among the volunteers, 22 are between 18 and 30 years old, 6 are between 31 and 40, and 7 are between 
41 and 50. All volunteers are with no personal or family history of colon cancer or polyps and all of them have not 
any gastrointestinal symptom All volunteers followed at least one of those 3 BPs and underwent virtual colonscopy 
CT scan. Some of them followed 2 or all 3 of those 3 BPs and underwent virtual colonscopy CT scan in different 
days. When the volunteers underwent CT scan, they got 2 series of scan at a time. One was for supine and another 
was for prone position. There were 60 cases, 120 series of scan, in total. An experienced radiologist reviewed 
electronic cleansing results of all cases utilizing the Viatronix V3D-Colon Module System. It was assumed that there 
was no polyp in all those healthy volunteers' colon. The radiologist reviewed electronic cleansed axial slices first to 
find any non-cleansed regions. Then, he flied through the 3-D colonic lumen model in order to search any cleansing 
artifact There were two kinds of artifacts. One was error removal of part of haustral fold. Another was the residue 
material with irregular shape in the lumen. The latter was due to inadequate cleansing. Finally, he reviewed the 
original multi-planar reformatted images for imaging artifacts unrelated to electronic cleansing which might be 
related to colon motion or scanner protocol. The radiologist was allowed to correlate supine and prone images to 
confirm the cleansing error. A case was recorded as inadequate cleansing if one of the following situations occurred. 

1. Inadequate cleansing situation I: There was at least one tagged region larger than 5 mm in diameter not 
being cleansed. (This usually was found when reviewed the cleansed axial images). 

2. Inadequate cleansing situation 2: There was at least one part of haustral fold thicker than 4 mm being 
removed. (This usually was found when flied through the 3-D lumen model). 

3. Inadequate cleansing situation 3: There were two or more areas of residual irregular material with their 
shortest diameter larger than 4mm in the colonic lumen. (This usually were found when flied through the 3- 
D lumen model) 

Fig. 16 lists the evaluation result By applying Fisher test, it was concluded that BP 1 was significantly better than 
both BP 2 (p = 0.01) and BP 3 (p = 0.001). BP 2 had not significantly different from the BP 3. This result showed 
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that the BP 1, the fluid diet BP, provided promising results with adequate tagging quality and electronic cleansing 
result all the time. The BP I may be the most feasible laxative free bowel preparation for virtual colonsocpy. 

The forgoing description focuses on oral intake of contrast agent. It is obvious that the electronic cleansing method 
is not limited to the oral intake contrast agent It should work also for other kinds of contrast agent administration 
unless the tagged material has intensity difference to its surrounding tissues. The intravenous injection procedure for 
virtual clonoscopy is an example. The contrast agent should not limited to medicine or specific chemicals either. It 
could be normal food or even natural water. For example, as shown in the [3], the natural water was utilized as the 
contrast agent for imaging bladder. In that application, the organ preparation time is only half an hour. More general, 
the electronic cleansing methods can be applied to any modality of images has the same feature in intensity. The 
maximum directional gradient approach can also be applied to any modality of medical images for extraction of 
tissue boundary layer with partial volume effects. 

The 3-D endoluminal view that was used in Viatronix V3D Colon Module was based on the volume rendering of the 
layer between the air and colon wall tissue. By reconstruction of the air/wall layer from the original residue/wall 
layer, it became feasible that the same rendering color map was able to apply to render entire cleansed colon lumen. 
The layer reconstruction was some kind of intensity transformation on the original volumetric data. This 
transformation was based on the information of both residue tagging and anatomy. The result of the cleansed 
volumetric images was the fusion of available information of both tagged residue and anatomy. In general, the 
fusion of information can be any kind of information that is related to the clinical application. The method of fusion 
of volumetric images with feature data is not limited to boundary layer reconstruction. The fusion method can be 
any transformation that maps feature data into the intensity of the volumetric images. The rendering method can also 
adaptively design for specific fusion technology and clinical requirement The direct method is to adjust the color 
map of rendering to allow different intensity range showing different color in the endoluminal view. 

In the application of laxative free virtual colonoscopy, computer assisted polyp detection (CAPD) results can be 
fused with the original volumetric images. The result of CAPD usually is a list of labeled regions for suspicious 
polyps and a list of likelihood that the region is related to be real polyp. One example of method to fuse the CAPD 
result with original volumetric images is to color code the suspicious regions in the volumetric images. Fig. 17 
demonstrates an example. Another approach to fused the CAPD results is to transform the intensity of suspicious 
region into certain range and adjust the color map for volume rendering to allow the suspicious region show 
different color from surrounding normal tissue in the 3-D endoluminal view. For example, the clean colon wall and 
the wall coated with tagged stool can be shown in different color by adjusting the volume rendering color maps. For 
MRI virtual cystoscopy [3], the bladder wall region could be extracted with presented maximum gradient approach. 
Then, the texture analysis can be applied to the bladder wall region. The texture indexes associated to voxels can be 
mapped back to the original volumetric data and volume rendering in the endoluminal view for facilitating detection 
of abnormality in the bladder wall. 
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The foregoing merely illustrates the principles of the invention. It will thus be appreciated that those skilled in the 
art will be able to devise numerous systems, apparatus and methods which, although not explicitly shown or 
described herein, embody the principles of the invention and are thus within the spirit and scope of the invention as 
defined by its claims. 

For example, the methods and systems described herein could be applied to virtually examine an animal or 
inanimate object. Besides the stated uses in the medical field, applications of the technique could be used to detect 
the contents of sealed objects which cannot be open or some of contents can be dyed to show contrast in the images. 

What is claimed is: 

1. A method for preparing volumetric images for virtual endoscopy was composite of the following steps: 

Organ preparation; 
Imaging the organ; 
Processing acquired images; 
Displaying images. 

2. The method as recited in claim 1 wherein the organ preparation represented administration of contrast 
agents. 

3. The method as recited in claim 1 wherein the organ preparation represented patient following specific diet. 

4. The method as recited in claim 1 wherein the organ preparation represented combination of administrating 
contrast agent and patient following specific diet 

5. The method as recited in claim 2 wherein the contrast agent was taken orally. 

6. The method as recited in claim 5 wherein the contrast agent was natural water that is taken by the patient 
half an hour prior to the MRI scan. The urine in the bladder forms a natural and non-invasive contrast 
agent. 

7. The method as recited in claim 2 wherein the contrast agent was injected through intravenous. 

8. The method as recited in claim 2 wherein the contrast agent was forced in the organ through a catherter. 

9. The method as recited in claim 1 wherein the imaging of the organ was undergone with CT scanner. 

10. The method as recited in claim 1 wherein the imaging of the organ was undergone with MRI. 

1 1 . The method as recited in claim 1 processing acquired images was composite of steps: 

Segmentation of regions of interest; 

Feature analysis to the regions of clinical interest; 

Fusion of original volumetric images and feature data. 

12. The method as recited in claim 1 1 wherein the segmentation of region of interest represented applying a 
low-level voxel classification first, then using anatomical knowledge to extract regions of clinical interest 
and to remove unrelated regions in the volumetric images. 

13. The method as recited in claim 12 wherein the extraction of regions of clinical interest represented the 
boundary layer extraction that covers the wall of the hollow organ. 
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14. The method as recited in claim 12 wherein the extraction of region of clinical interest represented the 
extraction of the region of the lumen of the organ. 

15. The method as recited in claim 1 1 wherein the feature analysis represented the computer assisted detection 
of abnormalities in the region of interest. 

16. The method as recited in claim 11 wherein (he fusion of original volumetric images and feature data 
represented the detection of the tagged region and replace the tagged region with other intensities of other 
tissue or material in the regions of interest. 

17. The method as recited in claim 11 wherein the fusion of original volumetric images and feature data 
represented the reconstruction of extracted boundary layer into another kind of boundary layer in the 
volumetric images. 

18. The method as recited in claim 11 wherein the fusion of original volumetric images and feature data 
represented transforming intensities of voxels in the original volumetric images that were in the detected 
suspicious abnormal regions into specific intensity range. 

19. The method as recited in claim 1 wherein displaying images represented volume rendering the volumetric 
images directly. 

20. The method as recited in claim I wherein displaying images represented volume rendering the fused 
volumetric images that was created as stated in claim 1 1. 

21. The method as recited in claim 1 wherein displaying images represented volume rendering the inner surface 
of the hollow organ based on either the original volumetric images or the fused volumetric images. 

22. The method as recited in claim 21 where volume rendering the inner surface of the organ represented 
adjusting the color map to display the suspicious area with different color within the context of the inner 
surface of the organ. 

23. A method for laxative free virtual colonoscopy was composite of the following steps: 

Bowel preparation without applying both oral laxative and suppository; 
Distending colon with specific material; 
Imaging colon; 
Processing acquired images. 
Displaying images. 

24. The method as recited in claim 23 wherein the bowel preparation represented administrating contrast agent. 

25. The method as recited in claim 24 wherein contrast agents were barium sulfate, diatrizoate meglumine, and 
diatrizoate sodium solution. 

26. The method as recited in claim 23 wherein the bowel preparation represented the combination of 
administrating of contrast agent and patient following specific diet. 

27. The method as recited in claim 26 wherein specific diet was any diet with low residue foods. 

28. The method as recited in claim 23 wherein distending colon with room air. 

29. The method as recited in claim 23 wherein distending colon with C0 2 . 

30. The method as recited in claim 23 wherein imaging colon with CT or MRI. 
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31. The method as recited in claim 23 wherein processing acquired images represented electronic cleansing 
tagged residue in the colon lumen. 

32. The method as recited in claim 23 wherein processing acquired images was composite of steps: 

Electronic cleansing tagged residue in the colon lumen; 
Automatic detection of suspicious polyps; 

Fusing polyp detection results into the electronic cleansed volumetric images. 

33. The method as recited in claim 31 wherein electronic cleansing tagged residue in the colon lumen was 
composite of steps: 

Segmentation of region of interest; 
Classification of tagged regions; 

Extraction of boundary layers between tagged residue and other tissues or materials; 
Cleansing tagged residue in the volumetric images; 

Transforming the boundary layer between tagged residue and colon wall into the boundary layer between 
air and colon wall in the volumetric images. 

34. The method as recited in claim 33 wherein segmentation of region of interest represented the detection and 
removal of regions of both lung and bone in the volumetric images by applying low-level voxel 
classification and using knowledge of anatomy. 

35. The method as recited in claim 33 wherein segmentation of region of interest represented the extraction of 
both air and tagged residue filled colon lumen by applying low-level voxel classification and using 
knowledge of anatomy. 

36. The method as recited in claim 33 wherein the classification of tagged regions represented the low-level 
voxel classification was applied to all tagged residue regions in the volumetric images. The number of 
created clusters was preset based on empirical knowledge. 

37. The method as recited in claim 33 wherein the extraction of boundary layers between tagged residue and 
other tissues or materials represented applying maximum directional gradient approach to extract boundary 
layer. 

38. The method as recited in claim 33 wherein the cleansing tagged residue in the volumetric images 
represented setting the intensity of non-boundary layer voxel in the tagged residue region into the average 
intensity of air voxels in the volumetric images. 

39. The method as recited in claim 33 wherein the transformation from the residue/wall boundary layer into the 
air/wall boundary layer was a linear transformation. 

40. The method as recited in claim 39 wherein the linear transforming was with a penalty term based on the 
maximum directional gradient value. 

41. The method as recited in claim 23 wherein the displaying images represented volume rendering the inner 
surface of the colon lumen with specific color map that could show colon wall voxels in different color if 
they were in different intensity ranges. 

42. A method for laxative free virtual colonscopy was composite of the following steps: 
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Administration of 4 doses of barium sulfates and 2 doses of Gastroview in 36 hours period prior to the CT 
scan; 

Patient following the fluid diet (foods are listed in Fig. 6) in the last 24 hours prior to the scan and 

following a suggested food list (See Fig. 4) in 2 days before the last 24 hours; 

Imaging the air/C02 rilled colon in both supine and prone position; 

Segmentation of region of interest; 

Electronic cleansing of tagged residue in the colon lumen; 

Fusion the results of automatic polyp detection if it was available; 

Display the fused volumetric images. 

43. A method for laxative free virtual colonscopy was composite of the following steps: 

Administration of 4 doses of barium sulfates and 2 doses of Gastroview in 36 hours period prior to the CT 
scan; 

Patient taking foods from a given Meal Kit (foods are listed in Fig. 5) in the last 24 hours prior to the scan 

and following a suggested food list (See Fig. 4) in 2 days before the last 24 hours; 

Imaging the air/C02 filled colon in both supine and prone position; 

Segmentation of region of interest; 

Electronic cleansing of tagged residue in the colon lumen; 

Fusion the results of automatic polyp detection if it was available; 

Display the volumetric images. 

44. A method for laxative free virtual colonscopy was composite of the following steps: 

Administration of 4 doses of barium sulfates and 2 doses of Gastroview in 36 hours period prior to the CT 
scan; 

Patient following a suggested food list (See Fig. 4) in 3 days prior to the CT scan; 

Imaging the air/C02 filled colon in both supine and prone position; 

Segmentation of region of interest; 

Electronic cleansing of tagged residue in the colon lumen; 

Fusion the results of automatic polyp detection if it was available; 

Display the volumetric images. 

45. A method for electronic cleansing tagged residue in a hollow organ for virtual endoscopy was composite of 
the following steps: 

Segmentation of the regions of tagged residue; 

Extraction of the boundary layer between tagged residue and wall of the organ; 

Transforming the intensity of voxels in the tagged region but not in the boundary layer into a preset 
intensity; 

Transforming the boundary layer between tagged residue and wall of the organ into another known type 
of boundary layer in the volumetric images; 
Displaying the transformed volumetric images. 



25 of 27 



46. A method as recited in claim 45 wherein Segmentation of the region of tagged residue represented a low- 
level voxel classification, 

47. A method as recited in claim 45 wherein extraction of the boundary layer between tagged residue and wall 
of the organ represented the maximum directional gradient approach. 

48. A method as recited in claim 47 wherein the maximum directional gradient approach was composite of 
steps: 

Selecting several discrete directions in the image grid space to uniformly cover all directions; 
Computing the first-order derivative along each selected direction for all voxels in the volumetric images; 
Determining the maximum absolute value of all directional first-order derivative for all voxels in the 
volumetric images. This was the so-called maximum directional gradient value (MDGV). 
Thresholding MDGVs. If the MDGV of a voxel was larger than a preset threshold, this voxel was in the 
boundary layer. 

49. The method recited in claim 48 wherein the computing the first-order derivative along each selected 
direction for each voxel must consider the voxel sizes along different directions. 

50. The method recited in claim 48 wherein the computing the first-order derivative along each selected 
direction for each voxel utilized different equation with longer range for reducing the affection of noise. 

51. The method recited in claim 48 wherein the preset threshold for thresholding MDGV was determined by 
computing voxels in a typical tissue boundary region in the volumetric images. 

52. The method recited in claim 45 wherein the tnmsforming non-boundary tagged voxels into a preset 
intensity, the preset intensity was the average intensity of a specific tissue or material in the volumetric 
images. 

53. The method recited in claim 45 wherein the transformation of the boundary tagged voxels was a linear 
transformation that flips intensity. 

54. The method recited in claim 45 wherein displaying the transformed volumetric images represented the 
volume rendering the inner surface of the hollow organ. The viewing point was located in the cleansed 
lumen. 

55. A method for extraction of layer of tissue or material boundary that have partial volume effect from 
medical images was composite of steps: 

Segmentation of the regions of interest; 
Extraction of the boundary layer; 

Transform the non-boundary layer voxels into a preset intensity in the volumetric images; 
Transform the boundary layer voxels into certain range of intensities in the volumetric images; 
Displaying the transformed volumetric images. 

56. The method recited in claim 54 wherein the extraction of the boundary layer represented the maximum 
directional gradient approach. 

57. The method recited in claim 54 wherein the transformation of the boundary layer voxels represented 
replacing the intensity with the MDGV of the voxel. 
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The method recited in claim 54 wherein the displaying of the transformed volumetric images represented 
the volume rendering of the transformed volumetric images. 

The method recited in claim 57 wherein the color map of volume rendering was adjusted to show different 
intensity range in different color. 
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